Papers with continual learning benchmarks
Continual Gradient Low-Rank Projection Fine-Tuning for LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Low-Rank Adaptation (LoRA) offers efficiency but constrains the model’s ability to learn new tasks and transfer knowledge due to its low-rank nature and reliance on explicit parameter constraints. |
| Approach: | They propose a training strategy that synergistically combines full and low-rank parameters and jointly updating within a unified low-ranked gradient subspace. |
| Outcome: | Extensive experiments on continual learning benchmarks show that GORP improves performance compared to state-of-the-art approaches. |